The Grand Challenge in Metamorphic Analysis

Malware detection is a crucial aspect of software security. Malware typically recur to a variety of disguise and concealing techniques in order to avoid detection. Metamorphism is the ability of a program to mutate its form yet keeping unchanged its functionality and therefore its danger in case of malware. A major challenge in this field is the development of general automatic/systematic detection techniques that are able to catch the possible variants of a metamorphic malware. We take the position that the key for handling metamorphism relies in a deeper understanding of the semantics of the metamorphic malware. By applying standard formal methods we aim at proving that metamorphic analysis is a special case of program analysis, where the object of computation is code interpreted as a mutational data structure. 1 Metamorphic Malware Analysis Detecting and neutralizing malware is a major challenge in computer security involving both sophisticated intrusion detection strategies and code manipulation tools and methods. Traditional misuse (or signature-based) malware detectors are syntactic in nature: They use pattern matching to compare the byte sequence comprising the body of the malware against a signature database [23]. Metamorphism emerged in the last decade as an effective strategy to foil misuse malware detectors. Metamorphic malware apply semantics preserving transformations (e.g. code obfuscation techniques) to modify their own code so that one instance of the malware bears very little resemblance to another instance even though semantically their functionality is the same. Thus, a metamorphic malware is a malware equipped with a metamorphic engine that takes the malware, or parts of it, as input and morphs it at run-time to a syntactically different but semantically equivalent variant, in order to foil signature matching. The quantity of metamorphic variants possible for a particular piece of malware makes it impractical to maintain a signature set that is large enough to cover most or all of these variants, making standard signature-based detection ineffective [5]. The reason for this vulnerability to metamorphism lies upon the purely syntactic nature of most exiting and commercial detectors that ignore program functionalities. Following this observation researchers began to develop detection techniques that take into account properties of the malware behavior instead of properties of its syntax. This naturally needs sophisticated program and behavioral analysis techniques, that rely upon known and new formal methods for reasoning about programs that mutate their code during execution. Code Variants Malware detection (behavioral analysis) Mutation insensitive analysis Standard analysis Kn ow led ge ab ou t co de tra nsf orm atio n s tra teg ies